Software defined networking (SDN) comprises a plurality of hosts in communication over a physical network infrastructure, each host having one or more virtual computing instances (VCIs) such as virtual machines (VMs) or containers that are connected to logical overlay networks that can span multiple hosts and are decoupled from the underlying physical network infrastructure. Though certain aspects herein are described with respect to VMs, it should be noted that the same aspects may be similarly used for other types of VCIs.
Virtualization software such as a hypervisor facilitates the creation and execution of VMs on a host. Hypervisors serve as an interface between VMs and the hardware resources of the hosts. A hypervisor can abstract processor, memory, storage, and networking resources of a host to allocate the host's hardware resources to multiple VMs.
For example, a host may have one or more physical CPUs (pCPUs). Each of the one or more pCPUs may be capable of operating at a particular frequency (e.g., measured in MHz, GHz, etc.). Further, different pCPUs may be capable of operating at the same or different frequencies. Similarly, each VM on a host may be defined as having one or more vCPUs. Different VMs may have the same or different number of vCPUs. A vCPU is seen by a VM as a physical CPU core by the VM's operating system. The hypervisor abstracts pCPU resources of the host into the vCPUs used by the VMs.
In particular, a hypervisor allocates pCPU resources of a host to a VM (i.e., to vCPUs of a VM), in part, using configuration metrics. The pCPU resources of the host are allocated using CPU resource allocation metrics such as a CPU reservation resource allocation metric (e.g., referred to as a reservation metric), a CPU limit resource allocation metric (e.g., referred to as a limit metric), and a CPU shares resource allocation metric (e.g., referred to as a shares metric) as explained below.
A CPU reservation resource allocation metric for a VM provides a guarantee of pCPU resources for the VM. In particular, a CPU reservation resource allocation metric is typically specified in MHz, and is a guarantee for clock cycles per second of pCPUs of a host (e.g., divided across any number of pCPUs of the host). As discussed, each of the pCPUs of a host may operate at a particular frequency. The sum of the frequencies of the pCPUs of the host corresponds to the total frequency or total clock cycles per second available at the host. A CPU reservation resource allocation metric guarantees a portion of the total frequency available at the host to the VM, meaning the portion of the total frequency available at the host for the VM is guaranteed as divided among the vCPUs of the VM. Accordingly, when a CPU reservation resource allocation metric provides a reservation for a certain amount of clock cycles for a certain VM, a CPU scheduler will guarantee the pCPU resources (e.g., 1 GHz of pCPU clock cycles per second of the host) to the VM provided by the CPU reservation resource allocation metric.
Typically, when the VM is not using all of its reserved pCPU resources, the pCPU resources are not wasted by the host, but rather the hypervisor can allocate them serially or concurrently to other VMs. Thus a CPU reservation resource allocation metric is used to provide a VM access to pCPUs of the host to support vCPUs in a committed environment (e.g., the pCPU is also reserved by other VMs). It will be appreciated that because VMs typically do not use all of the pCPU resources allocated by a hypervisor, a hypervisor may allocate pCPU resources to a plurality of VMs so long as the total workload for all VMs will not exceed the pCPU resources at any given time. It will be further appreciated that a CPU reservation resource allocation metric may also be specified in a percent of pCPU resources (e.g., a 50% reservation of clock cycles of pCPUs of a host). It will be appreciated that a pCPU may refer to a core of a CPU.
A CPU limit resource allocation metric is typically specified in MHz and sets an upper maximum amount of pCPU resources that can be allocated to a VM. More specifically, the CPU limit resource allocation metric prevents a VM from using more clock cycles per second of pCPUs of the host (e.g., divided across any number of pCPUs of the host) than the CPU limit resource allocation metric provides even if more clock cycles per second are not being used and are available. In this case, the VM's performance is restricted by the CPU limit resource allocation metric even though the host has further capacity. This is used to prevent a VM from using too much of a host's resources at any given time. It will be further appreciated that a CPU limit resource allocation metric may also be specified in a percent of pCPU resources.
A CPU shares resource allocation metric provides a number of shares of a VM. For example, a VM is typically configured with a certain number of shares (e.g., 1000 shares) by a hypervisor. In a default setting, each VM may be configured with an equal number of shares, but it will be appreciated that the number of shares can be allocated using a hypervisor as needed for the VMs (e.g., to prioritize a first VM over a second VM). The CPU shares resource allocation metric is used to govern CPU resource distribution as long as other resource allocation metrics are not violated. Thus, the CPU shares resource allocation metric provides a selection mechanism for access to the pCPUs by providing a relative importance level between VMs. In certain aspects, when there is a case of contention for pCPU resources, a first VM associated with a higher number of shares gets access to the pCPU resources over a second vCPU associated with a lower number of shares. In other aspects, when there is a case of contention for pCPU resources, vCPUs get access to the pCPU resources proportionality to the number of shares associated with each vCPU (e.g., vCPU1 of VM1 with an allocated 1000 shares gets half the amount of pCPU resources as vCPU2 of VM2 with an allocated 2000 shares).
The CPU resource allocation metrics described above can result in vCPU latency for VMs of about hundreds of milliseconds, meaning there may be hundreds of milliseconds or more of time in between when a VM requests use of a vCPU for processing a workload and when pCPU resources are actually made available to the vCPU for processing the workload. Further, the vCPUs also experience unbounded (or random) jitter as the latency may vary over time from hypervisor contexts. This is because the current CPU resource allocation metrics are based on overall utilization (throughput) of pCPUs of a host. For example, as discussed, the hypervisor may have overcommitted the pCPU resources of a host. In this case, the hypervisor may not be able to provide all VMs with the requested pCPU resources at a given time. It will be appreciated that this can cause poor performance (e.g., high latency) when processing a workload on a VM. It will be appreciated that this can lead to situations where certain VMs have to wait for an unreasonable amount of time for the pCPU resources. It will be further appreciated, that the above CPU resource allocation metrics do not provide a latency guarantee.
Such jitter and latency may not be suitable for executing certain workloads. For example, workloads in internet connected devices (e.g., IoT Edge gateway devices, etc.) and network functions virtualization (NFV) in data driven industries, such as the telecommunications industry, often require certain quality of service (QoS) standards (e.g., performance, availability, and reliability) to meet a certain service level agreement (SLA). In order to execute such workloads in VMs, this translates into needing a predictable responsiveness of such VMs with controlled latency and jitter that meets the SLA criteria. For example, SLA in the telecommunications industry for NFV often requires millisecond or even sub-millisecond CPU latency with predictable performance (e.g., an error ratio of one bit error in 106).
Current solutions for meeting certain SLAs for a workload executing in a VM include a CPU resource allocation that dedicates at least one pCPU in a host for each vCPU of the VM executing the workload. This is referred to as physical core pinning, and it is associated with certain drawbacks including higher associated costs and efficiency losses. Physical core pinning is associated with higher costs and efficiency losses, in part, because the dedicated pCPU can no longer be time-shared across multiple vCPUs such as across multiple VMs, forgoing a significant benefit to using vCPUs. Thus there exists a need to provide execution of workloads on a VM while meeting certain CPU QoS requirements (e.g., SLA QoS requirements of millisecond or sub-millisecond latency), without the need to pin a vCPU to a pCPU.